<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<html>

<!--

   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
-->

<head />
<body bgcolor="white">
<h1>Multi Cluster Replication</h1>
This package provides replication between HBase clusters.
<p>

<h2>Table Of Contents</h2>
<ol>
    <li><a href="#status">Status</a></li>
    <li><a href="#requirements">Requirements</a></li>
    <li><a href="#deployment">Deployment</a></li>
    <li><a href="#verify">Verifying Replicated Data</a></li>
</ol>

<p>
<a name="status">
<h2>Status</h2>
</a>
<p>
This package is experimental quality software and is only meant to be a base
for future developments. The current implementation offers the following
features:

<ol>
    <li>Master/Slave replication.</li>
    <li>Master/Master replication.</li>
    <li>Cyclic replication.</li>
    <li>Replication of scoped families in user tables.</li>
    <li>Start/stop replication stream.</li>
    <li>Supports clusters of different sizes.</li>
    <li>Handling of partitions longer than 10 minutes.</li>
    <li>Ability to add/remove slave clusters at runtime.</li>
    <li>MapReduce job to compare tables on two clusters</li>
</ol>
Please report bugs on the project's Jira when found.
<p>
<a name="requirements">
<h2>Requirements</h2>
</a>
<p>

Before trying out replication, make sure to review the following requirements:

<ol>
    <li>Zookeeper should be handled by yourself, not by HBase, and should
    always be available during the deployment.</li>
    <li>All machines from both clusters should be able to reach every
    other machine since replication goes from any region server to any
    other one on the slave cluster. That also includes the
    Zookeeper clusters.</li>
    <li>Both clusters should have the same HBase and Hadoop major revision.
    For example, having 0.90.1 on the master and 0.90.0 on the slave is
    correct but not 0.90.1 and 0.89.20100725.</li>
    <li>Every table that contains families that are scoped for replication
    should exist on every cluster with the exact same name, same for those
    replicated families.</li>
    <li>For multiple slaves, Master/Master, or cyclic replication version
    0.92 or greater is needed.</li>
</ol>

<p>
<a name="deployment">
<h2>Deployment</h2>
</a>
<p>

The following steps describe how to enable replication from a cluster
to another.
<ol>
    <li>Edit ${HBASE_HOME}/conf/hbase-site.xml on both cluster to add
    the following configurations:
        <pre>
&lt;property&gt;
  &lt;name&gt;hbase.replication&lt;/name&gt;
  &lt;value&gt;true&lt;/value&gt;
&lt;/property&gt;</pre>
      deploy the files, and then restart HBase if it was running.
    </li>
    <li>Run the following command in the master's shell while it's running
    <pre>add_peer 'ID' 'CLUSTER_KEY'</pre>
    The ID must be a short integer. To compose the CLUSTER_KEY, use the following template:
    <pre>hbase.zookeeper.quorum:hbase.zookeeper.property.clientPort:zookeeper.znode.parent</pre>
    This will show you the help to setup the replication stream between
    both clusters. If both clusters use the same Zookeeper cluster, you have
    to use a different <b>zookeeper.znode.parent</b> since they can't
    write in the same folder.
    </li>
    <li>
    Once you have a peer, you need to enable replication on your column families.
    One way to do it is to alter the table and to set the scope like this:
    <pre>
      disable 'your_table'
      alter 'your_table', {NAME => 'family_name', REPLICATION_SCOPE => '1'}
      enable 'your_table'
    </pre>
    Currently, a scope of 0 (default) means that it won't be replicated and a
    scope of 1 means it's going to be. In the future, different scope can be
    used for routing policies.
    </li>
    <li>To list all configured peers run the following command in the master's
    shell
    <pre>list_peers</pre> (as of version 0.92)
    </li>
    <li>To enable a peer that was previousy disabled, run the following command in the master's shell.
      <pre>enable_peer 'ID'</pre>
    </li>
    <li>To disable a peer, run the following command in the master's shell. This setting causes 
      HBase to stop sending the edits to that peer cluster, but it still keeps track of all the 
      new WALs that it will need to replicate if and when it is re-enabled.
    <pre>disable_peer 'ID'</pre>
    </li>
    <li>To remove a peer, use the following command in the master's shell.
      <pre>remove_peer 'ID'</pre>
    </li>
</ol>

You can confirm that your setup works by looking at any region server's log
on the master cluster and look for the following lines;

<pre>
Considering 1 rs, with ratio 0.1
Getting 1 rs from peer cluster # 0
Choosing peer 10.10.1.49:62020</pre>

In this case it indicates that 1 region server from the slave cluster
was chosen for replication.<br><br>
<p>

<a name="verify">
<h2>Verifying Replicated Data</h2>
</a>
<p>
Verifying the replicated data on two clusters is easy to do in the shell when
looking only at a few rows, but doing a systematic comparison requires more
computing power. This is why the VerifyReplication MR job was created, it has
to be run on the master cluster and needs to be provided with a peer id (the
one provided when establishing a replication stream) and a table name. Other
options let you specify a time range and specific families. This job's short
name is "verifyrep" and needs to be provided when pointing "hadoop jar" to the
hbase jar.
</p>

</body>
</html>
